Video displaying apparatus and controlling method thereof

ABSTRACT

Disclosed is a video displaying apparatus including: a projector screen; at least one camera disposed at a predetermined position of the projector screen to acquire a user&#39;s video; a communication unit receiving the other party&#39;s video from the other video terminal and transmitting the user&#39;s video acquired through at least one camera to the other video terminal; and a projector outputting the other party&#39;s video so that the other party&#39;s video received from the communication unit is displayed on the projector screen.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to and the benefit of Korean PatentApplication No. 10-2011-0022414 and 10-2010-0123756 filed in the KoreanIntellectual Property Office on Mar. 14, 2011, and Dec. 6, 2010, theentire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present invention relates to a video displaying apparatus matching aline of sight between a user and a camera and freely controlling a sizeof a video conference screen and a controlling method thereof.

BACKGROUND ART

A video conference system is a bidirectional communication system whichreceives videos and voices acquired from a plurality of terminals andoutputs the videos and voices to different terminals to exchangeinformation while each user (caller) using the plurality of terminalslooks at the other party's face.

In each terminal included in the video conference system, since a camerafor capturing a user's appearance which is installed at the upperportion of the monitor photographs a user's face video looking at themonitor where the other party's video is provided to provide thephotographed user's face video to the other party, the user's appearancedisplayed on the other's monitor always looks downward, such that a lineof sight does not coincide with each other.

Further, a mirror and a half mirror are obliquely installed at a regularangle so as to have a predetermined angle and the camera is installed atthe rear of the half mirror so as to watch the video reflected throughthe mirror with the user's naked eye, such that the position of theuser's line of sight and a focus axis of the camera may be configured tobe disposed on the same line as each other. However, in theconfiguration, since a mirror having a screen size so as to reflect thescreen output of the monitor displaying the video transmitted from theother party's terminal, a half mirror for re-reflecting the videoreflected by the mirror, and the like are additionally required on thefront of the monitor, there are inconveniences in that a spatial limitis generated and the position of the user's line of sight and the focusaxis of the camera need to coincide with each other by installing aseparate auxiliary device for fixing the mirror and the half mirrorbelow the monitor.

In addition, in each terminal included in the video conference system,it is difficult to configure a large screen by using the monitor-baseddisplay, the monitor has to be disposed in advance at a fixed place dueto a volume thereof, a lot of costs of the monitor are required inproportion to the screen size, and in the case where the videoconference system is configured by using a 3D video, an increase in thecost occurs according to the additional installation of the 3D monitor.

SUMMARY OF THE INVENTION

The present invention has been made in an effort to provide a videodisplaying apparatus matching a line of sight between a user and acamera by inserting one or more cameras in the center or severalportions of a display unit and a controlling method thereof.

Further, the present invention has been made in an effort to provide avideo displaying apparatus variably providing sizes of one or morevideos displayed on the display unit (for example, a screen) based on aprojector and providing a 3D video conference and a controlling methodthereof.

In addition, the present invention has been made in an effort to providea video displaying apparatus providing a synchronization functionbetween a projector and a camera in order to prevent a video (or light)outputted from the projector from being reflected on the camera orremoving the video (light) outputted from the projector included in thevideo photographed by the camera by a signal processing method and acontrolling method thereof.

An exemplary embodiment of the present invention provides a videodisplaying apparatus performing a video communication with another videoterminal, including: a projector screen; at least one camera disposed ata predetermined position of the projector screen to acquire a user'svideo; a communication unit receiving the other party's video from theother video terminal and transmitting the user's video acquired throughthe at least one camera to the other video terminal; and a projectoroutputting the other party's video so that the other party's videoreceived from the communication unit is displayed on the projectorscreen.

The video displaying apparatus may further include a synchronizationmodule removing the other party's video outputted from the projector inthe video acquired from the camera to acquire the user's video.

The video displaying apparatus may further include a line of sighttracking and controlling module controlling the position of the cameraso that the user's changed line of sight and a focus axis of the cameraare the same line as each other when the user's line of sight of thevideo displaying apparatus is changed.

The predetermined position of the projector screen on which the camerais installed may be the center of the projector screen or the screencenter on which the other party's video is displayed.

The video displaying apparatus may further include a speaker outputtinga voice corresponding to the other party's video displayed on theprojector screen.

When the camera is two or more and the user's line of sight is changed,the communication unit may transmit the user's video from the camerawhen the user's changed line of sight and the focus axis are disposed onthe same line as each other to the other video terminal.

Another exemplary embodiment of the present invention provides acontrolling method of a video displaying apparatus using a projector inorder to communicate with two or more the other parties, the methodincluding: displaying two or more the other party's videos on aprojector screen; detecting the other party's video corresponding to theuser's line of sight among the two or more the other party's videos;acquiring the user's video acquired from the camera corresponding to thedetected other side's video among two or more cameras installed on theprojector screen; and transmitting the acquired user's video to theother party's video displaying apparatus.

The controlling method may further include removing the other party'svideo outputted from the projector in the video acquired from the camerato acquire the user's video.

The present invention provides the following effects.

First, according to exemplary embodiments of the present invention, itis possible to perform a natural call while looking at the other partyin a video conference or a video calling by inserting one or morecameras at the center or several portions of a display unit to match aline of sight between a user and a camera.

Second, according to exemplary embodiments of the present invention, itis possible to variably control a screen during a video conference or avideo calling and perform a 3D video conference by variably providingsizes of one or more videos displayed on the screen based on a projectorand providing the 3D video conference.

Third, according to exemplary embodiments of the present invention, itis possible to dispose one or more cameras at any position on a screenand photograph video information including a user through thecorresponding camera by providing a synchronization function between aprojector and a camera in order to prevent a video (or light) outputtedfrom the projector from being reflected on the camera or removing thevideo (light) outputted from the projector included in the videophotographed by the camera by a signal processing method.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the drawings and the followingdetailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a videoconference system according to an exemplary embodiment of the presentinvention.

FIG. 2 is a block diagram illustrating a configuration of a videoconference terminal according to an exemplary embodiment of the presentinvention.

FIG. 3 is a flowchart illustrating a video conference method accordingto a first exemplary embodiment of the present invention.

FIG. 4 is a flowchart illustrating a communication process betweenterminals included in a video conference system according to a secondexemplary embodiment of the present invention.

FIG. 5 is a flowchart illustrating a video conference method accordingto a third exemplary embodiment of the present invention.

FIG. 6 is a flowchart illustrating a communication process betweenterminals included in a video conference system according to a fourthexemplary embodiment of the present invention.

FIG. 7 is a flowchart illustrating a video conference method accordingto a fifth exemplary embodiment of the present invention.

It should be understood that the appended drawings are not necessarilyto scale, presenting a somewhat simplified representation of variousfeatures illustrative of the basic principles of the invention. Thespecific design features of the present invention as disclosed herein,including, for example, specific dimensions, orientations, locations,and shapes will be determined in part by the particular intendedapplication and use environment.

In the figures, reference numbers refer to the same or equivalent partsof the present invention throughout the several figures of the drawing.

DETAILED DESCRIPTION

Hereinafter, exemplary embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings. Firstof all, we should note that in giving reference numerals to elements ofeach drawing, like reference numerals refer to like elements even thoughlike elements are shown in different drawings. In describing the presentinvention, well-known functions or constructions will not be describedin detail since they may unnecessarily obscure the understanding of thepresent invention. It should be understood that although exemplaryembodiment of the present invention are described hereafter, the spiritof the present invention is not limited thereto and may be changed andmodified in various ways by those skilled in the art.

Exemplary embodiments of the present invention may be implemented byvarious means. For example, the exemplary embodiments of the presentinvention may be implemented firmware, software, or a combinationthereof, or the like.

In the implementation by the hardware, a method according to exemplaryembodiments of the present invention may be implemented by applicationspecific integrated circuits (ASICs), digital signal processors (DSPs),digital signal processing devices (DSPDs), programmable logic devices(PLDs), field programmable gate arrays (FPGAs), processors, controllers,microcontrollers, microprocessors, or the like.

In the implementation using the firmware or the software, a methodaccording to exemplary embodiments of the present invention may beimplemented by modules, procedures, functions, or the like, that performfunctions or operations described above. Software codes are stored in amemory unit and may be driven by a processor. The memory unit isdisposed in or out the processor and may transmit and receive data toand from the well-known various units.

Throughout the specification, when a predetermined portion is describedto be “connected to” another portion, it includes a case where thepredetermined portion is electrically connected to the other portion bydisposing still another predetermined portion therebetween, as well as acase where the predetermined portion is directly connected to the otherportion. Also, when the predetermined portion is described to include apredetermined constituent element, it indicates that unless otherwisedefined, the predetermined portion may further include anotherconstituent element, not precluding the other constituent element.

The term module described in the present specification indicates asingle unit to process a predetermined function or operation and may beconfigured by hardware or software, or a combination of hardware andsoftware.

Specific terms are provided to help understandings of the presentinvention. The use of the specific terms may be changed into other formswithout departing from the technical idea of the present invention.

The present invention relates to a video conference terminal matching aline of sight between a user and a camera by inserting one or morecameras at the center or several portions of a display unit and acontrolling method thereof. In the present invention, it is possible tomatch a line of sight between a user and a camera by inserting one ormore cameras at the center or several potions of a display unit,variably provide sizes of one or more videos displayed on the displayunit (for example, a screen) based on a projector, provide a 3D videoconference, provide a synchronization function between the projector andthe camera in order to prevent the video outputted from the projectorfrom being reflected on the camera, or remove the video (light)outputted from the projector included in the video photographed by thecamera by a signal processing method.

Hereinafter, exemplary embodiments of the present invention will bedescribed in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram illustrating a configuration of a videoconference system 10 according to an exemplary embodiment of the presentinvention.

The video conference system 10 according to the exemplary embodiment ofthe present invention includes a server 100, a plurality of videoconference terminals 200, and a communication network 300 connecting theserver 100 and the plurality of video conference terminals 200 through awire/wireless communication system.

The server 100 performs a call connection (communication connection)function for a video conference (video calling) between the plurality ofvideo conference terminals 200 included in the video conference system10, a monitoring function of states of each video conference terminal(including a call state, a network state, and the like), a call endingfunction, and the like.

According to an exemplary embodiment, the server 100 may be configuredso that any video conference terminal 200 among the plurality of videoconference terminals 200 included in the video conference system 10performs the function of the corresponding server 100.

The video conference terminal 200 according to the exemplary embodimentof the present invention includes a communication unit 210, an inputunit 220, a storing unit 230, a controller 240, and an output unit 250as shown in FIG. 2.

The communication unit (or wireless communication module) 210 accordingto the exemplary embodiment of the present invention performs atransmitting and receiving function of voice and video data between thevideo conference terminal 200 and an external video conference terminal.

The communication unit 210 may include one or more components forperforming a wire/wireless communication with at least one videoconference terminal 200 included in the video conference system 10 or awire/wireless communication between the video conference terminal 200and a network in which the video conference terminal 200 is disposed. Inthis case, the communication unit 210 may include a module for wirelessInternet connection or a short range communication. Herein, as awireless Internet technique, a wireless LAN (WLAN), a wireless broadband(Wibro), a Wi-Fi, a world interoperability for microwave access (Wimax),a high speed downlink packet access (HSDPA), and the like may beincluded and as a short range communication technique, a bluetooth, aZigBee, an ultra wideband (UWB), a radio frequency identification(RFID), an infrared data association (IrDA), and the like may beincluded. A wire communication technique may include a universal serialbus (USB) communication and the like.

The communication unit 210 may perform communication with the server 100or the external video conference terminal through the communicationnetwork 300 based on a unique IP address which is preset (or randomlyset) in the video conference terminal 200.

The communication unit 210 transmits and receives signals with at leastone of a base station, an external terminal (including a videoconference terminal, a mobile terminal, and the like), and the server100 by using the communication network 300. Herein, the signals mayinclude various types of data according to the transmission andreception of a voice call signal, a video call signal, and/or atext/multimedia message.

The input unit 220 according to the exemplary embodiment of the presentinvention includes at least one microphone 221 and at least one camera222, as shown in FIG. 2.

The microphone 221 according to the exemplary embodiment of the presentinvention receives an external sound signal by a microphone in a callingmode, a recording mode, a voice recognition mode, a video calling mode,a video conference mode, and the like to process the external soundsignal to electrical voice data. The processed voice data may beoutputted through a speaker 253 included in the output unit 250 orconverted into a transmittable form by an external terminal andoutputted. The microphone 221 may apply various noise removal algorithmsin order to remove noise generated in a process of receiving theexternal sound signal.

The camera 222 according to the exemplary embodiment of the presentinvention processes a video frame of a still video, a moving video, orthe like acquired by at least one image sensor in a video calling mode,a photographing mode, a video conference mode, and the like. That is,the corresponding video data acquired by the image sensor are decoded soas to conform to each standard according to a CODEC. The video frameprocessed in the camera 222 may be displayed on the output unit 250 bythe control of the controller 240. The video frame processed in thecamera 222 may be stored in the storing unit 230 or transmitted to theoutside (including any video conference terminals included in the videoconference system 10) through the communication unit 210.

The camera 222 is installed at any position of the display unit 252included in the output unit 250. Herein, in the case where one camera222 is disposed at the display unit 252, the corresponding camera may bedisposed at a fixed position (for example, any one position of thecenter of the display unit 252, the center of the screen on which theother party's video is displayed, and a point at which the other party'sface (or eye) included in the other party's video is disposed) ormovably installed.

The input unit 220 receives a signal according to a button control bythe user or receives a command or a control signal generated by thecontrol, for example, touching/scrolling the displayed screen.

The input unit 220 may use various devices such as a keyboard, a keypad, a dome switch, a touch pad (constant pressure/electrostatic), atouch screen, a jog shuttle, a jog wheel, a jog switch, a mouse, astylus pen, a touch pen, a laser pointer, and the like. In this case,the input unit 220 receives a signal corresponding to the input by thevarious devices.

The storing unit 230 according to the exemplary embodiment of thepresent invention stores data and programs required when the videoconference terminal 200 operates. The storing unit 230 stores a signalprocessing algorithm for removing the video (light) outputted from aprojector 251 included in the output unit 250 from the videosphotographed through the camera 222.

The storage unit 230, for example, may include at least one storagemedium of a flash memory type, a hard disk type, a multimedia card microtype, a card type memory (for example, an SD or XD memory, or the like),a magnetic memory, a magnetic disk, an optical disk, a random accessmemory (RAM), a static random access memory (SPAM), a read-only memory(ROM), an electrically erasable programmable read-only memory (EEPROM),and a programmable read-only memory (PROM).

The storing unit 230 stores various kinds of management informationincluding video codec setting information of the corresponding videoconference terminal 200 (including video/audio/codec settinginformation, etc.), call standard setting information (including H.248,H.323, SIP, MGCP, intelligent network application protocol (INAP)), ISDNuser part (ISUP), transmission control application part (TCAP), MEGACO,V.35/RS-449/RS-530 setting information, international services digitalnetwork (ISDN), call basic setting information, etc.), network settinginformation (including network address translation (NAT), domain nameservice (DNS), IP address, QoS information, streaming information,simple network management protocol (SNMP), etc.), security settinginformation (including firewall setting information, etc.), and thelike.

The controller 240 according to the exemplary embodiment of the presentinvention controls overall operations of the video conference terminal200. For example, the controller 240 performs controlling and processingrelating to a voice calling, a data communication, a video calling, avideo conference, and the like. The controller 240 performs an encodingor decoding function for the video photographed through the camera 222or the received video.

The controller 240 includes a video conference control module 241, asynchronization module 242, and a line of sight tracking and controllingmodule 243, as shown in FIG. 2. Herein, the controller 240 may beconfigured so as to be included in the video conference terminal 200,configured as a separately independent device, or configured in a formembedded in the camera 222 or the projector 251 included in the outputunit 250.

The video conference control module 241 according to the exemplaryembodiment of the present invention manages the call connection relatedto the video calling (video conference) with at least any one of thevideo conference terminals. In the case where the video conferencecontrol module 241 normally communicates with at least any one of thevideo conference terminals, the video conference control module 241performs video/voice communication (or video conference) with at leastany one of the video conference terminals. That is, while at least anyone of the video conference terminals included in the video conferencesystem 10 and the video conference system operate, the video conferencecontrol module 241 processes the video and/or voice informationtransmitted from at least any one of the video conference terminals to apredetermined signal to output the processed signal through the outputunit 250.

The video conference control module 241 transmits the user's voiceacquired through the microphone 221 to the server 100 through thecommunication unit 210. The video conference control module 241transmits the user's video acquired through the camera 222 to the server100 through the communication unit 210.

Herein, the user's video means a video in which the video (light)outputted from the projector 251 included in the output unit 250 fromthe videos photographed through the camera 222 is removed (for example,the video in which the video outputted from the projector 251 is removedfrom the videos photographed through the camera 222 after the camera 222and the projector 251 are synchronized or the videos photographed by thecamera 222 through the signal processing process).

The video conference control module 241 outputs the user's voiceacquired through the microphone 221 or the other party's voicetransmitted from any video conference terminal received through thecommunication unit 210 through the speaker 253 included in the outputunit 250.

The video conference control module 241 displays one or more of theuser's video acquired through the camera 222 or one or more the otherparty's videos received through the communication unit 210, and picturein picture (PIP) data configured by merging the user's video and one ormore the other party's videos on the display unit 252 through theprojector 251 included in the output unit 250. The video conferencecontrol module 241 may receive the video mixed with the video of theplurality of video conference terminals through the communication unit210 and display the received mixed video on the display unit 252 throughthe projector 251. That is, when the video conference control module 241performs the video calling (video conference) with the plurality ofvideo conference terminals, the video conference control module 241 mayreceive the mixed video in the server 100 to output the received mixedvideo, or receive each video transmitted from each video conferenceterminal and mix (merge) the received video and the user's videoacquired through the camera 222 to output the mixed video.

When any video is displayed on the display unit 252 through theprojector 251 included in the output unit 250, the video conferencecontrol module 241 may display the video by variably changing the sizeof the displayed video and display the corresponding video on thedisplay unit 252 in a 3D mode by using one projector or a plurality ofprojectors 251.

The video conference control module 241 controls the operation of theplurality of cameras 222. That is, the video conference control module241 controls the operation of any camera 222 selected by the line ofsight tracking and controlling module 243 among the plurality of cameras222.

The synchronization module 242 according to the exemplary embodiment ofthe present invention synchronizes the camera 222 and the projector 251,in order to prevent the videos photographed through the camera 222 fromoverlapping with each other because the videos photographed through thecamera 222 provided at any position of the display unit 252 included inthe output unit 250 overlap with the video outputted from the projector251 included in the output unit 250. The synchronization module 242removes the video (light) outputted from the projector 251 from thevideos (including the videos (light) outputted from the projector 251)photographed through the camera 222 by using a pre-stored signalprocessing algorithm in the storing unit 230, in order to prevent thevideos photographed through the camera 222 from overlapping with eachother because the videos photographed through the camera 222 provided atany position of the display unit 252 included in the output unit 250overlap with the video outputted from the projector 251 included in theoutput unit 250. That is, by the control of the synchronization module242, the user's video photographed through the camera 222 may be a videoin which the video (light) outputted from the projector 251 is removed.

The line of sight tracking and controlling module 243 according to theexemplary embodiment of the present invention selects the correspondingcamera 222 coinciding with the user's line of sight among the pluralityof cameras 222 provided at the plurality of positions of the displayunit 252 included in the output unit 250. That is, the line of sighttracking and controlling module 243 recognizes the user (person) amongobjects photographed through the plurality of cameras 222 by usingvarious object recognition algorithms pre-stored in the storing unit230, checks a position of the user's eyes of the recognized user's face,calculates viewing angles between the checked position of the user'seyes and a focus of each camera 222 (for example, including a horizontalangle, a vertical angle, etc. with respect to an X axis and a Y axis),and selects the camera 222 having the smallest viewing angle among thecalculated viewing angles (or the camera coinciding with the line ofsight between the user and the camera).

The line of sight tracking and controlling module 243 checks whether theuser's line of sight is changed in real time and then, when the user'sline of sight is changed and the user's line of sight is changed fromthe previously selected camera to another camera, the cameracorresponding to the user's changed line of sight (or the most similarto the user's changed line of sight among the plurality of cameras) isselected.

The line of sight tracking and controlling module 243 calculates aviewing angle between one camera 222 provided at any position of thedisplay unit 252 included in the output unit 250 and the user's line ofsight and moves the camera 222 so that the user's line of sightcoincides with the camera 222 based on the calculated viewing angle.Herein, when the user's line of sight coincides with the camera 222, theposition of the user's line of sight may be disposed on the same line asa focus axis of the camera (or the center of a camera lens).

The line of sight tracking and controlling module 243 checks whether theuser's line of sight is changed in real time and then, when the user'sline of sight is changed, the line of sight tracking and controllingmodule 243 controls the camera 222 so as to move the position of thecamera 222 in accordance with the user's changed line of sight.

The output unit 250 according to the exemplary embodiment of the presentinvention includes a projector 251, a display unit 252, and a speaker253, as shown in FIG. 2.

The projector 251 according to the exemplary embodiment of the presentinvention is movable or portable and configured so as to be fixed at apredetermined position including a support.

The display unit 252 according to the exemplary embodiment of thepresent invention displays the video (light) outputted from theprojector 251. Herein, the display unit 252 may be any screen and mayalso be all available display devices capable of displaying the videooutputted from the projector 251.

The display unit 252 includes any support member and one or more cameras222 may be disposed at the support member. Herein, one or more cameras222 disposed at any position of the display unit 252 may be disposed soas to form the same plane as the surface of the display unit 252. Whenthe plurality of cameras 222 are disposed on the display unit 252, thepositions of the plurality of cameras 222 on the display unit 252 may bedisplayed on the display unit 252 by the control of the controller 240.

The positions of one or more cameras 222 disposed at any position of thedisplay unit 252 may be controlled by the control of the line of sighttracking and controlling module 243. Camera angles of one or morecameras 222 may be controlled in upper/lower/left/right directions. Thatis, one or more cameras 222 are configured so as to control a tilt or apan.

The speaker 253 according to the exemplary embodiment of the presentinvention outputs the user's voice or the other party's voice.

The communication network 300 according to the exemplary embodiment ofthe present invention connects the server 100 and one or more videoconference terminals 200 to each other. The communication network 300includes a hub, a bridge, a gateway, a router, and the like.

FIG. 3 is a flowchart illustrating a video conference method accordingto a first exemplary embodiment of the present invention.

Hereinafter, this diagram will be described with reference to FIGS. 1 to3.

First, when the video conference mode (video calling mode) is set, thevideo conference control module 241 included in a first video conferenceterminal 200 requests a call connection with a second video conferenceterminal 200 to any server 100 connected to each other through thecommunication network 300. The video conference control module 241receives normal call connection success information with the secondvideo conference terminal 200 transmitted from the server 100 throughthe communication unit (S110).

The input unit 220 receives user's voice and video information throughthe microphone 221 and the camera 222, respectively. Herein, the camera222 is provided at a predetermined position of the display unit 252included in the output unit 250. In this case, the camera 222 isprovided at the position of the display unit 252 (for example, thecenter of the display unit 252 or the screen center on which the otherparty's video is displayed) where the user's line of sight of the firstvideo conference terminal 200 coincides with the camera 222 and isconfigured so as to be not fixed but freely movable.

The synchronization module 242 synchronizes the projector 251 and thecamera 222, in order to prevent the videos photographed through thecamera 222 from overlapping with each other because the videophotographed through the camera 222 overlaps with the video outputtedfrom the projector 251. The synchronization module 242 removes the video(light) outputted from the projector 251 from the videos photographedthrough the camera 222 by using a pre-stored signal processing algorithmin the storing unit 230 (S130).

The video conference control module 241 transmits the user's voiceand/or the user's video (for example, the video in which the lightoutputted from the projector 251 is removed from the videos photographedthrough the camera 222 after the camera 222 and the projector 251 aresynchronized or the videos photographed by the camera 222) to the server100 through the communication unit 210 (S140).

The video conference control module 241 receives a second user's voice(the other party's voice) and/or the second user's video (the otherparty's video) acquired from the second video conference terminaltransmitted from the server 100 through the communication unit 210.

The video conference control module 241 displays one or more of theuser's video acquired from the first video conference terminal, thereceived other side's video, and the PIP data merging the user's videoand the other party's video on the display unit 252 through theprojector 251.

The video conference control module 241 outputs the user's voice or theother party's voice acquired from the first video conference terminalthrough the speaker 253.

As described above, when the video conference mode (video calling mode)is set, the video conference control module 241 transmits the user'svideo and/or the user's voice to the second video conference terminalthrough the communication network 300 to share the user's video and/orthe user's voice with the other party's video conference terminalinvolved in the video conference system 10 (S150).

FIG. 4 is a flowchart illustrating a communication process betweenterminals included in a video conference system according to a secondexemplary embodiment of the present invention.

First, when the video conference mode (video calling mode) is set(selected), the first video conference control module 241 included inthe first video conference terminal 200 performs a call connection(video calling or communication connection) with a second videoconference terminal 200 through any server 100 connected to each otherby using the communication network 300 (SP21).

The first video conference terminal 200 receives the first user's voiceand the first video information through the microphone 221 and the firstcamera 222 included in the first input unit 220. Herein, the firstcamera 222 is provided at a predetermined position of the first displayunit 252 included in the first output unit 250. In this case, the firstcamera 222 is provided at the position of the first display unit 252where the user's line of sight of the first video conference terminal200 coincides with the first camera 222 (for example, the center of thefirst display unit 252 or the screen center on which the other party'svideo is displayed) and is configured so as to be not fixed but freelymovable (SP22).

The synchronization module 242 synchronizes the first projector 251 andthe first camera 222 or removes the video (light) outputted from thefirst projector 251 from the videos photographed through the firstcamera 222 by using a pre-stored signal processing algorithm in thefirst storing unit 230 (SP23).

The first video conference control module 241 transmits the first user'svoice and/or the first user's video (for example, the video in which thelight outputted from the first projector 251 is removed from the videosphotographed through the first camera 222 after the first projector 251and the first camera 222 are synchronized or the videos photographed bythe first camera 222) to the server 100 through the first communicationunit 210. The server 100 transmits the first user's voice and/or thefirst user's video transmitted from the first video conference controlmodule 241 to the second video conference control module 241 included inthe second video conference terminal 200 (SP24).

The second video conference terminal 200 receives the second user'svoice and the second video information through the second microphone 221and the second camera 222 included in the second input unit 220,respectively. The second synchronization module 242 included in thesecond video conference terminal 200 synchronizes the second projector251 and the second camera 222, or removes the video (light) outputtedfrom the second projector 251 from the videos photographed through thesecond camera 222 by using a pre-stored signal processing algorithm inthe second storing unit 230 (SP25).

The second video conference control module 241 transmits the seconduser's voice and/or the second user's video (for example, the video inwhich the light outputted from the second projector 251 is removed fromthe videos photographed through the second camera 222 after the secondcamera 222 and the second projector 251 are synchronized or the videosphotographed by the second camera 222) to the server 100 through thesecond communication unit 210. The server 100 transmits the seconduser's voice (the other party's voice) and/or the second user's video(the other party's video) transmitted from the second video conferencecontrol module 241 to the first video conference control module 241(SP26).

The first video conference control module 241 receives the other party'svoice and/or the other party's video acquired from the second videoconference terminal transmitted from the server 100 through the firstcommunication unit 210.

The second video conference control module 241 receives the first user'svoice and/or the first user's video transmitted from the server 100(SP27).

The first video conference control module 241 displays one or more ofthe first user's video acquired from the first video conferenceterminal, the received other side's video (the second user's video), andthe PIP data configured by merging the first user's video and the otherparty's video on the display unit 252 through the first projector 251.The first video conference control module 241 outputs the first user'svoice or the other party's voice (the second user's voice) acquired fromthe first video conference terminal through the first speaker 253(SP28).

The second video conference control module 241 displays one or more ofthe second user's video acquired from the second video conferenceterminal, the received first user's video, and the PIP data configuredby merging the second user's video and the first user's video on thesecond display unit 252 through the second projector 251. The secondvideo conference control module 241 outputs the second user's voice orthe first user's voice acquired from the second video conferenceterminal through the second speaker 253 (SP29).

FIG. 5 is a flowchart illustrating a video conference method accordingto a third exemplary embodiment of the present invention.

Hereinafter, this diagram will be described with reference to FIGS. 1,2, and 5.

First, when the video conference mode (video calling mode) is set, thevideo conference control module 241 included in the first videoconference terminal 200 performs a call connection among the pluralityof video conference terminals involved in the video conference system 10through any server 100 connected to each other by using thecommunication network 300 (S310).

The line of sight tracking and controlling module 243 selects thecorresponding camera 222 coinciding with the user's line of sight amongthe plurality of cameras 222 provided at the plurality of positions ofthe display unit 252. That is, the line of sight tracking andcontrolling module 243 calculates viewing angles between the pluralityof cameras 222 and the user's line of sight (for example, including ahorizontal angle, a vertical angle, and the like with respect to an Xaxis and a Y axis) and selects the camera having the smallest viewingangle among the calculated viewing angles. Herein, the plurality ofcameras are configured so as to be not fixed but freely movable (S320).

The input unit 220 receives the user's voice and the video informationthrough the microphone 221 and the previously selected camera 222,respectively.

The synchronization module 242 synchronizes the projector 251 and thecamera 222, in order to prevent the videos photographed through thecamera 222 from overlapping with each other because the videophotographed through the camera 222 overlaps with the video outputtedfrom the projector 251. The synchronization module 242 removes the video(light) outputted from the projector 251 from the videos photographedthrough the selected camera 222 by using a pre-stored signal processingalgorithm in the storing unit 230 (S330).

The video conference control module 241 transmits the user's voiceand/or the user's video (for example, the video in which the lightoutputted from the projector 251 is removed from the videos photographedthrough the selected camera 222 after the camera 222 and the projector251 are synchronized or the videos photographed by the selected camera222) to the server 100 through the communication unit 210 (S340).

The video conference control module 241 receives the video mixed withthe video information of each of the plurality of video conferenceterminals involved in the video conference system 10 transmitted fromthe server 100 through the communication unit 210. The video conferencecontrol module 241 outputs the user's voice (or the other party's voice)acquired from any video conference terminal transmitted from the server100 through the communication unit 210.

The video conference control module 241 displays the received mixedvideo on the display unit 252 through the projector 251.

The video conference control module 241 outputs the user's voice or theother party's voice acquired from the first video conference terminalthrough the speaker 253 (S350).

The line of sight tracking and controlling module 243 checks whether theuser's line of sight is changed in real time (S360).

When the user's line of sight is changed and then, the user's line ofsight is changed from the previously selected camera to another camera,the line of sight tracking and controlling module 243 selects thecorresponding camera 222 in accordance with the user's changed line ofsight (S370).

The video conference control module 241 temporally stops the operationof the previously selected camera and controls the operation of thecorresponding camera 222 coinciding with the user's changed line ofsight to receive the video from the corresponding camera 222 andtransmit the received video to the server 100 through the communicationunit 210 (S380).

FIG. 6 is a flowchart illustrating a communication process betweenterminals included in a video conference system according to a fourthexemplary embodiment of the present invention.

First, when the video conference mode (video calling mode) is set(selected), the first video conference control module 241 included inthe first video conference terminal 200 performs a call connection(video calling or communication connection) with the plurality of videoconference terminals involved in the video conference system 10 throughany server 100 connected to each other by using the communicationnetwork 300 (SP41).

The first line of sight tracking and controlling module 243 selects thecorresponding camera 222 coinciding with the user's line of sight amongthe plurality of cameras 222 provided at the plurality of positions ofthe first display unit 252. That is, the first line of sight trackingand controlling module 243 calculates viewing angles between theplurality of first cameras 222 and the user's line of sight (forexample, including a horizontal angle, a vertical angle, and the likewith respect to an X axis and a Y axis) and selects the camera havingthe smallest viewing angle among the calculated viewing angles. Herein,the plurality of cameras are configured so as to be not fixed but freelymovable (SP42).

The first input unit 220 receives the user's voice and the videoinformation through the first microphone 221 and the previously selectedcamera 222, respectively.

The first synchronization module 242 synchronizes the first projector251 and the first camera 222, or removes the video (light) outputtedfrom the first projector 251 from the videos photographed through theselected camera 222 by using a pre-stored signal processing algorithm inthe first storing unit 230.

Other plurality of video conference terminals included in the videoconference system 10 receive user's voice and video information by usinga microphone, a synchronization module, and a camera, and the likeprovided at each video conference terminal (SP43).

The first video conference control module 241 transmits the first user'svoice and/or the first user's video (for example, the video in which thelight outputted from the first projector 251 is removed from the videosphotographed through the camera 222 after the first projector 251 andthe first camera 222 are synchronized or the videos photographed by thecamera 222) to the server 100 through the first communication unit 210.

Each of the other video conference terminals included in the videoconference system 10 transmits the received user's voice and videoinformation to the server 100 (SP44).

The server 100 receives the videos transmitted from the plurality ofvideo conference terminals included in the video conference system 10,mixes the plurality of received videos, and transmits the mixed video tothe plurality of video conference terminals, respectively (SP45).

The first video conference control module 241 receives the mixed videotransmitted from the server 100 through the first communication unit210. The first video conference control module 241 receives the user'svoice (or the other party's voice) acquired from any video conferenceterminal transmitted from the server 100 through the first communicationunit 210.

Each of the other video conference terminals included in the videoconference system 10 receive the mixed video transmitted from the server100 and/or the user's voice (the other party's voice) acquired from anyvideo conference terminal (SP46).

The first video conference control module 241 displays the receivedmixed video on the first display unit 252 through the first projector251. The first video conference control module 241 outputs the user'svoice or the other party's received voice acquired from the first videoconference terminal through the first speaker 253.

Each of the other video conference terminals included in the videoconference system 10 outputs the received mixed video or voice (SP47).

The first line of sight tracking and controlling module 243 checkswhether the user's line of sight is changed in real time (SP48).

When the user's line of sight is changed and then, the user's line ofsight is changed from the previously selected camera to another camera,the first line of sight tracking and controlling module 243 selects thecorresponding camera 222 in accordance with the user's changed line ofsight. That is, when the user's line of sight is changed, the first lineof sight tracking and controlling module 243 selects the camera 222 ofwhich the focus axis is disposed on the same line as the user's changedline of sight, among the plurality of cameras.

The first video conference control module 241 temporally stops theoperation of the previously selected camera and controls the operationof the corresponding camera 222 coinciding with the user's changed lineof sight to receive the video from the corresponding camera 222 andtransmit the received video to the server 100 through the communicationunit 210 (SP49).

FIG. 7 is a flowchart illustrating a video conference method accordingto a fifth exemplary embodiment of the present invention.

Hereinafter, this diagram will be described with reference to FIGS. 1,2, and 7.

First, when the video conference mode (video calling mode) is set, thevideo conference control module 241 included in the first videoconference terminal 200 performs a call connection with at least onevideo conference terminal involved in the video conference system 10through any server 100 connected to each other by using thecommunication network 300 (S510).

The line of sight tracking and controlling module 243 calculates viewingangles between the camera 222 provided at any position of the displayunit 252 and the user's line of sight (for example, including ahorizontal angle, a vertical angle, and the like with respect to an Xaxis and a Y axis) and moves the camera 222 so as for the camera 222 tocoincide with the user's line of sight based on the calculated viewingangle. Herein, the camera 222 is configured so as to be not fixed butfreely movable (S520).

The input unit 220 receives the user's voice and video informationthrough the microphone 221 and the camera 222 moved so as to coincidewith the above user's line of sight.

The synchronization module 242 synchronizes the projector 251 and thecamera 222, in order to prevent the videos photographed through thecamera 222 from overlapping with each other because the videophotographed through the camera 222 overlaps with the video outputtedfrom the projector 251. The synchronization module 242 removes the video(light) outputted from the projector 251 from the videos photographedthrough the selected camera 222 by using a pre-stored signal processingalgorithm in the storing unit 230 (S530).

The video conference control module 241 transmits the user's voiceand/or the user's video (for example, the video in which the lightoutputted from the projector 251 is removed from the videos photographedthrough the camera 222 after the camera 222 and the projector 251 aresynchronized or the videos photographed by the camera 222) to the server100 through the communication unit 210 (S540).

The video conference control module 241 receives at least one user'svoice (the other party's voice) and/or at least one user's video (atleast one of the other party's video) acquired from at least one videoconference terminal involved in the video conference system 10transmitted from the server 100 through the communication unit 210.

The video conference control module 241 displays one or more of theuser's video acquired from the first video conference terminal, at leastone of the received other side's video, and the PIP data configured bymerging the user's video and at least one of the other party's video onthe display unit 252 through the projector 251.

The video conference control module 241 outputs the user's voice or atleast one of the other party's voice acquired from the first videoconference terminal through the speaker 253 (S550).

The line of sight tracking and controlling module 243 checks whether theuser's line of sight is changed in real time (S560).

When the user's line of sight is changed, the line of sight tracking andcontrolling module 243 controls the camera 222 so as to coincide withthe user's changed line of sight to move the position of the camera 222.That is, when the user's line of sight is changed, the line of sighttracking and controlling module 243 move the position of the camera 222so that the focus axis of the camera 222 is disposed on the same line asthe user's changed line of sight (S570).

The video conference control module 241 receives the video from thecamera 222 of which the position is changed so as to coincide with theuser's changed line of sight and transmits the received video to theserver 100 through the communication unit 210 (S580).

As described above, the exemplary embodiments have been described andillustrated in the drawings and the specification. The exemplaryembodiments were chosen and described in order to explain certainprinciples of the invention and their practical application, to therebyenable others skilled in the art to make and utilize various exemplaryembodiments of the present invention, as well as various alternativesand modifications thereof. As is evident from the foregoing description,certain aspects of the present invention are not limited by theparticular details of the examples illustrated herein, and it istherefore contemplated that other modifications and applications, orequivalents thereof, will occur to those skilled in the art. Manychanges, modifications, variations and other uses and applications ofthe present construction will, however, become apparent to those skilledin the art after considering the specification and the accompanyingdrawings. All such changes, modifications, variations and other uses andapplications which do not depart from the spirit and scope of theinvention are deemed to be covered by the invention which is limitedonly by the claims which follow.

What is claimed is:
 1. A video displaying apparatus performing a videocommunication with another video terminal, comprising: a projectorscreen; at least one camera disposed at a predetermined position of theprojector screen to acquire a user's video; a communication unitreceiving the other party's video from the another video terminal andtransmitting the user's video acquired through at least one camera tothe other video terminal; a projector outputting the other party's videoso that the other party's video received from the communication unit isdisplayed on the projector screen; a synchronization module using asignal processing algorithm that removes the other party's videooutputted from the projector from the video acquired from the camera toacquire the user's video.
 2. The apparatus of claim 1, furthercomprising: a storing unit storing the signal processing algorithm. 3.The apparatus of claim 1, further comprising: a line of sight trackingand controlling module recognizing the user among objects photographedthrough the camera (1) by using object recognition algorithms used tocheck a position of the user's eyes of a recognized user's face, (2) bycalculating viewing angles between the checked position of the user'seyes and (3) by controlling the position and focus of the camera so thatthe user's changed line of sight and a focus axis of the camera are thesame line as each other when the user's line of sight of the videodisplaying apparatus is changed.
 4. The apparatus of claim 1, whereinthe predetermined position of the projector screen on which the camerais installed is the center of the projector screen or the screen centeron which the other party's video is displayed.
 5. The apparatus of claim1, further comprising: a speaker outputting a voice corresponding to theother party's video displayed on the projector screen.
 6. The apparatusof claim 1, wherein when the camera is two or more and the user's lineof sight is changed, the communication unit transmits the user's videofrom the camera when the user's changed line of sight and the focus axisare disposed on the same line as each other to the other video terminal.7. A controlling method of a video displaying apparatus using aprojector in order to communicate with two or more the other parties,the method comprising: displaying two or more the other party's videoson a projector screen; detecting the other party's video correspondingto the user's line of sight among the two or more the other party'svideos; acquiring the user's video acquired from the cameracorresponding to the detected other side's video among two or morecameras installed on the projector screen; and removing the otherparty's video outputted from the projector from the video acquired fromthe camera to acquire the user's video using a signal processingalgorithm; and transmitting the acquired user's video to the otherparty's video displaying apparatus.
 8. The method of claim 7, furthercomprising: removing the other party's video outputted from theprojector in the video acquired from the camera to acquire the user'svideo.
 9. A video displaying apparatus configured to perform a videocommunication with another video terminal, the video displayingapparatus comprising: a projector screen; at least one camera disposedat a predetermined position at the projector screen to acquire a user'svideo; a communication unit configured to receive the other party'svideo from the another video terminal and configured to transmit theuser's video acquired through at least one camera to the other videoterminal; a projector configured to output the other party's video sothat the other party's video received from the communication unit isdisplayed on the projector screen; and a line of sight tracking andcontrolling module configured (1) to recognize the user among objectsphotographed through the camera by using object recognition algorithmsused to check a position of the user's eyes of a recognized user's face,(2) to calculate viewing angles between the checked position of theuser's eyes, and (3) to control the position and focus of the camera sothat the user's changed line of sight and a focus axis of the camera arethe same line as each other when the user's line of sight of the videodisplaying apparatus is changed.
 10. The apparatus of claim 9, furthercomprising: a synchronization module configured to use a signalprocessing algorithm that removes the other party's video outputted fromthe projector from the video acquired from the camera to acquire theuser's video.
 11. The apparatus of claim 9, further comprising: aspeaker configured to output a voice corresponding to the other party'svideo displayed on the projector screen.
 12. The apparatus of claim 9,further comprising: a storing unit configured to store the objectrecognition algorithms.
 13. The apparatus of claim 9, wherein thepredetermined position of the projector screen on which the camera isinstalled is the center of the projector screen or the screen center onwhich the other party's video is displayed.
 14. The apparatus of claim10, further comprising a storing unit configured to store the signalprocessing algorithm.